• Principled Out-of-Distribution Detection via Multiple Testing

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Principled Out-of-Distribution Detection via Multiple Testing Akshayaa Magesh , Venugopal V . Veeravalli , Anirban Roy , Susmit Jha 24(378 1 35, 2023. Abstract We study the problem of out-of-distribution OOD detection , that is , detecting whether a machine learning ML model's output can be trusted at inference time . While a number of tests for OOD detection have been proposed in prior work , a formal framework for studying this problem is lacking . We propose a definition for the notion of OOD that includes both the input distribution and the ML model , which provides insights for the construction

  • Scaling Up Models and Data with t5x and seqio

    Updated: 2023-12-31 22:02:12
    Scaling up training datasets and model parameters have benefited neural network-based language models, but also present challenges like distributed compute, input data bottlenecks and reproducibility of results. We introduce two simple and scalable software libraries that simplify these issues: t5x enables training large language models at scale, while seqio enables reproducible input and evaluation pipelines. These open-source libraries have been used to train models with hundreds of billions of parameters on multi-terabyte datasets. Configurations and instructions for T5-like and GPT-like models are also provided. The libraries can be found at https://github.com/google-research/t5x and https://github.com/google/seqio.

  • Operator learning with PCA-Net: upper and lower complexity bounds

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Operator learning with PCA-Net : upper and lower complexity bounds Samuel Lanthaler 24(318 1 67, 2023. Abstract PCA-Net is a recently proposed neural operator architecture which combines principal component analysis PCA with neural networks to approximate operators between infinite-dimensional function spaces . The present work develops approximation theory for this approach , improving and significantly extending previous work in this direction : First , a novel universal approximation result is derived , under minimal assumptions on the underlying operator and the data-generating distribution .

  • Mixed Regression via Approximate Message Passing

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Mixed Regression via Approximate Message Passing Nelvin Tan , Ramji Venkataramanan 24(317 1 44, 2023. Abstract We study the problem of regression in a generalized linear model GLM with multiple signals and latent variables . This model , which we call a matrix GLM , covers many widely studied problems in statistical learning , including mixed linear regression , max-affine regression , and mixture-of-experts . The goal in all these problems is to estimate the signals , and possibly some of the latent variables , from the observations . We propose a novel approximate message passing AMP algorithm for

  • MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us MARLlib : A Scalable and Efficient Multi-agent Reinforcement Learning Library Siyi Hu , Yifan Zhong , Minquan Gao , Weixun Wang , Hao Dong , Xiaodan Liang , Zhihui Li , Xiaojun Chang , Yaodong Yang 24(315 1 23, 2023. Abstract A significant challenge facing researchers in the area of multi-agent reinforcement learning MARL pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and algorithm combinations , while obviating the need to consider compatibility issues . In this paper , we present MARLlib , a library designed to address the

  • TorchOpt: An Efficient Library for Differentiable Optimization

    Updated: 2023-12-31 22:02:12
    Differentiable optimization algorithms often involve expensive computations of various meta-gradients. To address this, we design and implement TorchOpt, a new PyTorch-based differentiable optimization library. TorchOpt provides an expressive and unified programming interface that simplifies the implementation of explicit, implicit, and zero-order gradients. Moreover, TorchOpt has a distributed execution runtime capable of parallelizing diverse operations linked to differentiable optimization tasks across CPU and GPU devices. Experimental results demonstrate that TorchOpt achieves a 5.2× training time speedup in a cluster. TorchOpt is open-sourced at https://github.com/metaopt/torchopt and has become a PyTorch Ecosystem project.

  • Hard-Constrained Deep Learning for Climate Downscaling

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Hard-Constrained Deep Learning for Climate Downscaling Paula Harder , Alex Hernandez-Garcia , Venkatesh Ramesh , Qidong Yang , Prasanna Sattegeri , Daniela Szwarcman , Campbell Watson , David Rolnick 24(365 1 40, 2023. Abstract The availability of reliable , high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation and to guide rapid responses to extreme events . Forecasting models are limited by computational costs and , therefore , often generate coarse-resolution predictions . Statistical downscaling , including super-resolution

  • Avalanche: A PyTorch Library for Deep Continual Learning

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Avalanche : A PyTorch Library for Deep Continual Learning Antonio Carta , Lorenzo Pellegrini , Andrea Cossu , Hamed Hemati , Vincenzo Lomonaco 24(363 1 6, 2023. Abstract Continual learning is the problem of learning from a nonstationary stream of data , a fundamental issue for sustainable and efficient training of deep neural networks over time . Unfortunately , deep learning libraries only provide primitives for offline training , assuming that model's architecture and data are fixed . Avalanche is an open source library maintained by the ContinualAI non-profit organization that extends PyTorch by

  • Fast Expectation Propagation for Heteroscedastic, Lasso-Penalized, and Quantile Regression

    Updated: 2023-12-31 22:02:12
    , , Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fast Expectation Propagation for Heteroscedastic , Lasso-Penalized , and Quantile Regression Jackson Zhou , John T . Ormerod , Clara Grazian 24(314 1 39, 2023. Abstract Expectation propagation EP is an approximate Bayesian inference ABI method which has seen widespread use across machine learning and statistics , owing to its accuracy and speed . However , it is often difficult to apply EP to models with complex likelihoods , where the EP updates do not have a tractable form and need to be calculated using methods such as multivariate numerical quadrature . These methods increase run time and

  • Discovering Salient Neurons in deep NLP models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Discovering Salient Neurons in deep NLP models Nadir Durrani , Fahim Dalvi , Hassan Sajjad 24(362 1 40, 2023. Abstract While a lot of work has been done in understanding representations learned within deep NLP models and what knowledge they capture , work done towards analyzing individual neurons is relatively sparse . We present a technique called Linguistic Correlation Analysis to extract salient neurons in the model , with respect to any extrinsic property , with the goal of understanding how such knowledge is preserved within neurons . We carry out a fine-grained analysis to answer the following

  • Differentially Private Hypothesis Testing for Linear Regression

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Differentially Private Hypothesis Testing for Linear Regression Daniel G . Alabi , Salil P . Vadhan 24(361 1 50, 2023. Abstract In this work , we design differentially private hypothesis tests for the following problems in the multivariate linear regression model : testing a linear relationship and testing for the presence of mixtures . The majority of our hypothesis tests are based on differentially private versions of the F$-statistic for the multivariate linear regression model framework . We also present other differentially private tests---not based on the F$-statistic---for these problems . We

  • Optimal Parameter-Transfer Learning by Semiparametric Model Averaging

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Optimal Parameter-Transfer Learning by Semiparametric Model Averaging Xiaonan Hu , Xinyu Zhang 24(358 1 53, 2023. Abstract In this article , we focus on prediction of a target model by transferring the information of source models . To be flexible , we use semiparametric additive frameworks for the target and source models . Inheriting the spirit of parameter-transfer learning , we assume that different models possibly share common knowledge across parametric components that is helpful for the target predictive task . Unlike existing parameter-transfer approaches , which need to construct auxiliary

  • MAUVE Scores for Generative Models: Theory and Practice

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us MAUVE Scores for Generative Models : Theory and Practice Krishna Pillutla , Lang Liu , John Thickstun , Sean Welleck , Swabha Swayamdipta , Rowan Zellers , Sewoong Oh , Yejin Choi , Zaid Harchaoui 24(356 1 92, 2023. Abstract Generative artificial intelligence has made significant strides , producing text indistinguishable from human prose and remarkably photorealistic images . Automatically measuring how close the generated data distribution is to the target distribution is central to diagnosing existing models and developing better ones . We present MAUVE , a family of comparison measures between

  • Beyond Spectral Gap: The Role of the Topology in Decentralized Learning

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Beyond Spectral Gap : The Role of the Topology in Decentralized Learning Thijs Vogels , Hadrien Hendrikx , Martin Jaggi 24(355 1 31, 2023. Abstract In data-parallel optimization of machine learning models , workers collaborate to improve their estimates of the model : more accurate gradients allow them to use larger learning rates and optimize faster . In the decentralized setting , in which workers communicate over a sparse graph , current theory fails to capture important aspects of real-world behavior . First , the spectral gap' of the communication graph is not predictive of its empirical

  • Prediction Equilibrium for Dynamic Network Flows

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Prediction Equilibrium for Dynamic Network Flows Lukas Graf , Tobias Harks , Kostas Kollias , Michael Markl 24(310 1 33, 2023. Abstract We study a dynamic traffic assignment model , where agents base their instantaneous routing decisions on real-time delay predictions . We formulate a mathematically concise model and define dynamic prediction equilibrium DPE in which no agent can at any point during their journey improve their predicted travel time by switching to a different route . We demonstrate the versatility of our framework by showing that it subsumes the well-known full information and

  • A Unified Experiment Design Approach for Cyclic and Acyclic Causal Models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Unified Experiment Design Approach for Cyclic and Acyclic Causal Models Ehsan Mokhtarian , Saber Salehkaleybar , AmirEmad Ghassami , Negar Kiyavash 24(354 1 31, 2023. Abstract We study experiment design for unique identification of the causal graph of a simple SCM , where the graph may contain cycles . The presence of cycles in the structure introduces major challenges for experiment design as , unlike acyclic graphs , learning the skeleton of causal graphs with cycles may not be possible from merely the observational distribution . Furthermore , intervening on a variable in such graphs does not

  • Dimension Reduction and MARS

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Dimension Reduction and MARS Yu Liu LIU , Degui Li , Yingcun Xia 24(309 1 30, 2023. Abstract The multivariate adaptive regression spline MARS is one of the popular estimation methods for nonparametric multivariate regression . However , as MARS is based on marginal splines , to incorporate interactions of covariates , products of the marginal splines must be used , which often leads to an unmanageable number of basis functions when the order of interaction is high and results in low estimation efficiency . In this paper , we improve the performance of MARS by using linear combinations of the

  • Nevis'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Nevis'22 : A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research Jorg Bornschein , Alexandre Galashov , Ross Hemsley , Amal Rannen-Triki , Yutian Chen , Arslan Chaudhry , Xu Owen He , Arthur Douillard , Massimo Caccia , Qixuan Feng , Jiajun Shen , Sylvestre-Alvise Rebuffi , Kitty Stacpoole , Diego de las Casas , Will Hawkins , Angeliki Lazaridou , Yee Whye Teh , Andrei A . Rusu , Razvan Pascanu , Marc’Aurelio Ranzato 24(308 1 77, 2023. Abstract A shared goal of several machine learning communities like continual learning , meta-learning and transfer learning , is to design

  • Group SLOPE Penalized Low-Rank Tensor Regression

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Group SLOPE Penalized Low-Rank Tensor Regression Yang Chen , Ziyan Luo 24(352 1 30, 2023. Abstract This article aims to seek a selection and estimation procedure for a class of tensor regression problems with multivariate covariates and matrix responses , which can provide theoretical guarantees for model selection in finite samples . Considering the frontal slice sparsity and low-rankness inherited in the coefficient tensor , we formulate the regression procedure as a group SLOPE penalized low-rank tensor optimization problem based on an orthogonal decomposition , namely TgSLOPE . This procedure

  • Modular Regression: Improving Linear Models by Incorporating Auxiliary Data

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Modular Regression : Improving Linear Models by Incorporating Auxiliary Data Ying Jin , Dominik Rothenhäusler 24(351 1 52, 2023. Abstract This paper develops a new framework , called modular regression , to utilize auxiliary information such as variables other than the original features or additional data sets in the training process of linear models . At a high level , our method follows the routine : i decomposing the regression task into several sub-tasks , ii fitting the sub-task models , and iii using the sub-task models to provide an improved estimate for the original regression problem .

  • Fast Screening Rules for Optimal Design via Quadratic Lasso Reformulation

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fast Screening Rules for Optimal Design via Quadratic Lasso Reformulation Guillaume Sagnol , Luc Pronzato 24(307 1 32, 2023. Abstract The problems of Lasso regression and optimal design of experiments share a critical property : their optimal solutions are typically sparse , i.e . only a small fraction of the optimal variables are non-zero . Therefore , the identification of the support of an optimal solution reduces the dimensionality of the problem and can yield a substantial simplification of the calculations . It has recently been shown that linear regression with a squared ell_1$-norm

  • Instance-Dependent Generalization Bounds via Optimal Transport

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Instance-Dependent Generalization Bounds via Optimal Transport Songyan Hou , Parnian Kassraie , Anastasis Kratsios , Andreas Krause , Jonas Rothfuss 24(349 1 51, 2023. Abstract Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks . Since such bounds often hold uniformly over all parameters , they suffer from over-parametrization and fail to account for the strong inductive bias of initialization and stochastic gradient descent . As an alternative , we propose a novel optimal transport interpretation of the generalization problem . This

  • Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queries

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queries Matteo Sesia , Stefano Favaro , Edgar Dobriban 24(348 1 80, 2023. Abstract This paper develops conformal inference methods to construct a confidence interval for the frequency of a queried object in a very large discrete data set , based on a sketch with a lower memory footprint . This approach requires no knowledge of the data distribution and can be combined with any sketching algorithm , including but not limited to the renowned count-min sketch , the count-sketch , and variations thereof . After

  • Linear Partial Monitoring for Sequential Decision Making: Algorithms, Regret Bounds and Applications

    Updated: 2023-12-31 22:02:12
    Partial monitoring is an expressive framework for sequential decision-making with an abundance of applications, including graph-structured and dueling bandits, dynamic pricing and transductive feedback models. We survey and extend recent results on the linear formulation of partial monitoring that naturally generalizes the standard linear bandit setting. The main result is that a single algorithm, information-directed sampling (IDS), is (nearly) worst-case rate optimal in all finite-action games. We present a simple and unified analysis of stochastic partial monitoring, and further extend the model to the contextual and kernelized setting.

  • Two Sample Testing in High Dimension via Maximum Mean Discrepancy

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Two Sample Testing in High Dimension via Maximum Mean Discrepancy Hanjia Gao , Xiaofeng Shao 24(304 1 33, 2023. Abstract Maximum Mean Discrepancy MMD has been widely used in the areas of machine learning and statistics to quantify the distance between two distributions in the p$-dimensional Euclidean space . The asymptotic property of the sample MMD has been well studied when the dimension p$ is fixed using the theory of U-statistic . As motivated by the frequent use of MMD test for data of moderate high dimension , we propose to investigate the behavior of the sample MMD in a high-dimensional

  • Large data limit of the MBO scheme for data clustering: convergence of the dynamics

    Updated: 2023-12-31 22:02:12
    We prove that the dynamics of the MBO scheme for data clustering converge to a viscosity solution to mean curvature flow. The main ingredients are (i) a new abstract convergence result based on quantitative estimates for heat operators and (ii) the derivation of these estimates in the setting of random geometric graphs. To implement the scheme in practice, two important parameters are the number of eigenvalues for computing the heat operator and the step size of the scheme. The results of the current paper give a theoretical justification for the choice of these parameters in relation to sample size and interaction width.

  • Pivotal Estimation of Linear Discriminant Analysis in High Dimensions

    Updated: 2023-12-31 22:02:12
    We consider the linear discriminant analysis problem in the high-dimensional settings. In this work, we propose PANDA(PivotAl liNear Discriminant Analysis), a tuning insensitive method in the sense that it requires very little effort to tune the parameters. Moreover, we prove that PANDA achieves the optimal convergence rate in terms of both the estimation error and misclassification rate. Our theoretical results are backed up by thorough numerical studies using both simulated and real datasets. In comparison with the existing methods, we observe that our proposed PANDA yields equal or better performance, and requires substantially less effort in parameter tuning.

  • A PDE approach for regret bounds under partial monitoring

    Updated: 2023-12-31 22:02:12
    In this paper, we study a learning problem in which a forecaster only observes partial information. By properly rescaling the problem, we heuristically derive a limiting PDE on Wasserstein space which characterizes the asymptotic behavior of the regret of the forecaster. Using a verification type argument, we show that the problem of obtaining regret bounds and efficient algorithms can be tackled by finding appropriate smooth sub/supersolutions of this parabolic PDE.

  • A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data sets

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data sets Jacques Wainer 24(341 1 34, 2023. Abstract his paper presents a Bayesian model , called the Bayesian Bradley Terry BBT model , for comparing multiple algorithms on multiple data sets based on any metric . The model is an extension of the Bradley Terry model , which tracks the number of wins each algorithm has on different data sets . Unlike frequentist methods such as Demsar tests on mean rank or multiple pairwise Wilcoxon tests , the Bayesian approach provides a more nuanced understanding of the algorithms’

  • High-Dimensional Inference for Generalized Linear Models with Hidden Confounding

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us High-Dimensional Inference for Generalized Linear Models with Hidden Confounding Jing Ouyang , Kean Ming Tan , Gongjun Xu 24(296 1 61, 2023. Abstract Statistical inferences for high-dimensional regression models have been extensively studied for their wide applications ranging from genomics , neuroscience , to economics . However , in practice , there are often potential unmeasured confounders associated with both the response and covariates , which can lead to invalidity of standard debiasing methods . This paper focuses on a generalized linear regression framework with hidden confounding and

  • Augmented Transfer Regression Learning with Semi-non-parametric Nuisance Models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Augmented Transfer Regression Learning with Semi-non-parametric Nuisance Models Molei Liu , Yi Zhang , Katherine P . Liao , Tianxi Cai 24(293 1 50, 2023. Abstract We develop an augmented transfer regression learning ATReL approach that introduces an imputation model to augment the importance weighting equation to achieve double robustness for covariate shift correction . More significantly , we propose a novel semi-non-parametric SNP construction framework for the two nuisance models . Compared with existing doubly robust approaches relying on fully parametric or fully non-parametric machine learning

  • Topological Hidden Markov Models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Topological Hidden Markov Models Adam B Kashlak , Prachi Loliencar , Giseon Heo 24(340 1 49, 2023. Abstract The Hidden Markov Model is a classic modelling tool with a wide swath of applications . Its inception considered observations restricted to a finite alphabet , but it was quickly extended to multivariate continuous distributions . In this article , we further extend the Hidden Markov Model from mixtures of normal distributions in d$-dimensional Euclidean space to general Gaussian measure mixtures in locally convex topological spaces , and hence , we christen this method the Topological Hidden

  • A Unified Analysis of Multi-task Functional Linear Regression Models with Manifold Constraint and Composite Quadratic Penalty

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Unified Analysis of Multi-task Functional Linear Regression Models with Manifold Constraint and Composite Quadratic Penalty Shiyuan He , Hanxuan Ye , Kejun He 24(291 1 69, 2023. Abstract This work studies the multi-task functional linear regression models where both the covariates and the unknown regression coefficients called slope functions are curves . For slope function estimation , we employ penalized splines to balance bias , variance , and computational complexity . The power of multi-task learning is brought in by imposing additional structures over the slope functions . We propose a

  • Low Tree-Rank Bayesian Vector Autoregression Models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Low Tree-Rank Bayesian Vector Autoregression Models Leo L Duan , Zeyu Yuwen , George Michailidis , Zhengwu Zhang 24(286 1 35, 2023. Abstract Vector autoregression has been widely used for modeling and analysis of multivariate time series data . In high-dimensional settings , model parameter regularization schemes inducing sparsity yield interpretable models and achieved good forecasting performance . However , in many data applications , such as those in neuroscience , the Granger causality graph estimates from existing vector autoregression methods tend to be quite dense and difficult to interpret ,

  • Near-Optimal Weighted Matrix Completion

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Near-Optimal Weighted Matrix Completion Oscar López 24(283 1 40, 2023. Abstract Recent work in the matrix completion literature has shown that prior knowledge of a matrix's row and column spaces can be successfully incorporated into reconstruction programs to substantially benefit matrix recovery . This paper proposes a novel methodology that exploits more general forms of known matrix structure in terms of subspaces . The work derives reconstruction error bounds that are informative in practice , providing insight to previous approaches in the literature while introducing novel programs with reduced

  • Dimensionality Reduction and Wasserstein Stability for Kernel Regression

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Dimensionality Reduction and Wasserstein Stability for Kernel Regression Stephan Eckstein , Armin Iske , Mathias Trabs 24(334 1 35, 2023. Abstract In a high-dimensional regression framework , we study consequences of the naive two-step procedure where first the dimension of the input variables is reduced and second , the reduced input variables are used to predict the output variable with kernel regression . In order to analyze the resulting regression errors , a novel stability result for kernel regression with respect to the Wasserstein distance is derived . This allows us to bound errors that

  • The Bayesian Learning Rule

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us The Bayesian Learning Rule Mohammad Emtiyaz Khan , Håvard Rue 24(281 1 46, 2023. Abstract We show that many machine-learning algorithms are specific instances of a single algorithm called the Bayesian learning rule . The rule , derived from Bayesian principles , yields a wide-range of algorithms from fields such as optimization , deep learning , and graphical models . This includes classical algorithms such as ridge regression , Newton's method , and Kalman filter , as well as modern deep-learning algorithms such as stochastic-gradient descent , RMSprop , and Dropout . The key idea in deriving such

  • Sparse Markov Models for High-dimensional Inference

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Sparse Markov Models for High-dimensional Inference Guilherme Ost , Daniel Y . Takahashi 24(279 1 54, 2023. Abstract Finite-order Markov models are well-studied models for dependent finite alphabet data . Despite their generality , application in empirical work is rare when the order d$ is large relative to the sample size n$ e.g . d mathcal{O n Practitioners rarely use higher-order Markov models because 1 the number of parameters grows exponentially with the order , 2 the sample size n$ required to estimate each parameter grows exponentially with the order , and 3 the interpretation is often

  • Weisfeiler and Leman go Machine Learning: The Story so far

    Updated: 2023-12-31 22:02:12
    In recent years, algorithms and neural architectures based on the Weisfeiler–Leman algorithm, a well-known heuristic for the graph isomorphism problem, have emerged as a powerful tool for machine learning with graphs and relational data. Here, we give a comprehensive overview of the algorithm’s use in a machine-learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph and node representation learning, discuss recent extensions, and outline the algorithm’s connection to (permutation-)equivariant neural architectures. Moreover, we give an overview of current applications and future directions to stimulate further research.

  • Learning Conditional Generative Models for Phase Retrieval

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Learning Conditional Generative Models for Phase Retrieval Tobias Uelwer , Sebastian Konietzny , Alexander Oberstrass , Stefan Harmeling 24(332 1 28, 2023. Abstract Reconstructing images from magnitude measurements is an important and difficult problem arising in many research areas , such as X-ray crystallography , astronomical imaging and more . While optimization-based approaches often struggle with the non-convexity and non- linearity of the problem , learning-based approaches are able to produce reconstructions of high quality for data similar to a given training dataset . In this work , we

  • Fair Data Representation for Machine Learning at the Pareto Frontier

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fair Data Representation for Machine Learning at the Pareto Frontier Shizhou Xu , Thomas Strohmer 24(331 1 63, 2023. Abstract As machine learning powered decision-making becomes increasingly important in our daily lives , it is imperative to strive for fairness in the underlying data processing . We propose a pre-processing algorithm for fair data representation via which supervised learning results in estimations of the Pareto frontier between prediction error and statistical disparity . In particular , the present work applies the optimal affine transport to approach the post-processing Wasserstein

  • The Power of Contrast for Feature Learning: A Theoretical Analysis

    Updated: 2023-12-31 22:02:12
    : Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us The Power of Contrast for Feature Learning : A Theoretical Analysis Wenlong Ji , Zhun Deng , Ryumei Nakada , James Zou , Linjun Zhang 24(330 1 78, 2023. Abstract Contrastive learning has achieved state-of-the-art performance in various self-supervised learning tasks and even outperforms its supervised counterpart . Despite its empirical success , theoretical understanding of the superiority of contrastive learning is still limited . In this paper , under linear representation settings , i we provably show that contrastive learning outperforms the standard autoencoders and generative adversarial

  • Reproducing Kernels and New Approaches in Compositional Data Analysis

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Reproducing Kernels and New Approaches in Compositional Data Analysis Binglin Li , Changwon Yoon , Jeongyoun Ahn 24(327 1 34, 2023. Abstract Compositional data , such as human gut microbiomes , consist of non-negative variables where only the relative values of these variables are available . Analyzing compositional data requires careful treatment of the geometry of the data . A common geometrical approach to understanding such data is through a regular simplex . The majority of existing approaches rely on log-ratio or power transformations to address the inherent simplicial geometry . In this work ,

  • Be More Active! Understanding the Differences Between Mean and Sampled Representations of Variational Autoencoders

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Be More Active Understanding the Differences Between Mean and Sampled Representations of Variational Autoencoders Lisa Bonheme , Marek Grzes 24(324 1 30, 2023. Abstract The ability of Variational Autoencoders to learn disentangled representations has made them appealing for practical applications . However , their mean representations , which are generally used for downstream tasks , have recently been shown to be more correlated than their sampled counterpart , on which disentanglement is usually measured . In this paper , we refine this observation through the lens of selective posterior collapse ,

  • Semi-Supervised Off-Policy Reinforcement Learning and Value Estimation for Dynamic Treatment Regimes

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Semi-Supervised Off-Policy Reinforcement Learning and Value Estimation for Dynamic Treatment Regimes Aaron Sonabend-W , Nilanjana Laha , Ashwin N . Ananthakrishnan , Tianxi Cai , Rajarshi Mukherjee 24(323 1 86, 2023. Abstract Reinforcement learning RL has shown great promise in estimating dynamic treatment regimes which take into account patient heterogeneity . However , health-outcome information , used as the reward for RL methods , is often not well coded but rather embedded in clinical notes . Extracting precise outcome information is a resource-intensive task , so most of the available

  • Higher-Order Spectral Clustering Under Superimposed Stochastic Block Models

    Updated: 2023-12-31 22:02:12
    Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Data DMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Higher-Order Spectral Clustering Under Superimposed Stochastic Block Models Subhadeep Paul , Olgica Milenkovic , Yuguo Chen 24(320 1 58, 2023. Abstract Higher-order motif structures and multi-vertex interactions are becoming increasingly important in studies of functionalities and evolution patterns of complex networks . To elucidate the role of higher-order structures in community detection over networks , we introduce a Superimposed Stochastic Block Model SupSBM The model is based on a random graph framework in which certain higher-order structures or subgraphs are generated through an independent

  • Best Data Visualization Projects of 2023

    Updated: 2023-12-29 08:53:04
    Membership Tutorials Courses Projects Newsletter Become a Member Log in Visualization best-of Best Data Visualization Projects of 2023 By Nathan Yau Data continues on its upwards trajectory and with it comes the importance of visualization . Many charts were made in 2023. If I liked something , it was on FlowingData These are my ten favorites from the . year Alvin Chang , for The Pudding 24 hours in an invisible epidemic This was a standout for me . I mess with data from the American Time Use Survey pretty much every year and Alvin’s project still caught me off guard . See the Project On FlowingData Lauren Leatherby , for The New York Times How a Vast Demographic Shift Will Reshape the World Population data . It’s another dataset we’ve seen many times , but I enjoyed the focus on age

  • ✚ Visualization Tools and Learning Resources, December 2023 Roundup

    Updated: 2023-12-28 19:30:26
    , Membership Tutorials Courses Projects Newsletter Become a Member Log in Members Only Visualization Tools and Learning Resources , December 2023 Roundup December 28, 2023 Topic The Process roundup Welcome to The Process the newsletter for FlowingData members where we look closer at how the charts get made . I’m Nathan Yau . Every month I collect useful tools and resources to help you make better charts . Here’s the last roundup of 2023. To access this issue of The Process , you must be a . member If you are already a member , log in here See What You Get The Process is a weekly newsletter on how visualization tools , rules , and guidelines work in practice . I publish every Thursday . Get it in your inbox or read it on FlowingData . You also gain unlimited access to hundreds of hours

  • ✚ Fun With Data

    Updated: 2023-12-21 19:30:58
    Membership Tutorials Courses Projects Newsletter Become a Member Log in Members Only Fun With Data December 21, 2023 Topic The Process fun Welcome to The Process the newsletter for FlowingData members that looks closer at how the charts get made . I’m Nathan Yau . With a few days left until Christmas , I suspect you have other things to do , but here are some fun things to do with data in case you’re looking for a distraction to pull you towards the . weekend To access this issue of The Process , you must be a . member If you are already a member , log in here See What You Get The Process is a weekly newsletter on how visualization tools , rules , and guidelines work in practice . I publish every Thursday . Get it in your inbox or read it on FlowingData . You also gain unlimited access to

  • Holiday special: why data storytelling matters

    Updated: 2023-12-18 15:35:00
    It’s a different kind of podcast this week: Simon and Alberto talk about Alberto’s latest book, ⁠The Art of Insight,⁠ why data journalism is still a dream job and our approaches to working with numbers to tell stories. Find out what books got us here – and what we care about most, when it comes … Continue reading →

  • Unwrapping Enhanced Interactivity for Calendar and Circle Packing Charts in AnyChart 8.12.0

    Updated: 2023-12-07 13:46:06
    Sales : 1 888 845-1211 USA or 44 20 7193 9444 Europe customer login Toggle navigation Products AnyChart AnyStock AnyMap AnyGantt Mobile Qlik Extension Features Resources Business Solutions Technical Integrations Chartopedia Tutorials Support Company About Us Customers Success Stories More Testimonials News Download Buy Now Search News » News » Unwrapping Enhanced Interactivity for Calendar and Circle Packing Charts in AnyChart 8.12.0 Unwrapping Enhanced Interactivity for Calendar and Circle Packing Charts in AnyChart 8.12.0 December 7th , 2023 by AnyChart Team As the magic holiday season approaches , we’re thrilled to unwrap our latest gift of the year — AnyChart 8.12.0 the newest version of our powerful JavaScript charting library Packed with various bug fixes and improvements ,

  • “Hungary is a data journalism superpower”

    Updated: 2023-12-04 23:36:11
    Attila Bátorfy⁠ is a data journalist operating in Viktor Orbán’s Hungary, heading up ATLO and pioneering the field in the country as a teacher and practitioner. Find out why he believes Hungary is the country to watch for data storytelling in this new episode of the Data Journalism Podcast. Music by ⁠⁠TwoTone⁠⁠, based on data … Continue reading →

Current Feed Items | Previous Months Items

Nov 2023 | Oct 2023 | Sep 2023 | Aug 2023 | Jul 2023 | Jun 2023